There are numerous products, services, business applications and uses, both at the commercial and individual consumer level, that would benefit from a cost-effective and efficient system and processes having the capability to up-scale video from smaller sizes and/or lower resolution to larger sizes and better resolution. One example of a rapidly growing commercial use for such products is evident from the continued growth of the HD television (HDTV) market, for which an increasing amount of current video and TV content is being created to take advantage of the HD display capabilities of such display devices. However, most pre-existing video or TV content made over the last century does not exist in a format or at a level of resolution quality that can take advantage of the display capabilities of HD televisions. For example, while current HDTV supports 700-1080 lines of resolution, most older and pre-existing movie or TV content owned by the various studios, distributors, or publishers, which include “re-runs”of older TV series and movies were filmed and designed to be broadcast, distributed, and displayed using “over-the-air” broadcasting at a mere 220-330 lines of resolution. Current hardware solutions to try to improve and display higher resolution and quality content are available but are typically very slow to process and can be very expensive to implement. In addition, most existing solutions merely replicate existing lines of resolution to give the illusion of a fuller picture, without improving the quality of the actual resolution.
Fractal geometry makes it possible to represent complex images using an elegant mathematical expression, which in most cases serves as a more compact alternative to the original raw image. Much like vectors, fractals have the inherent ability to scale infinitely without any discernable loss in image quality. That is, to say, by substituting a variable within the fractal expression of an object, a new larger object can be created when the fractal expression is reinterpreted to display an image on the screen. A simple example of up-scaling from a lower resolution image 100a to a high resolution image 100b is illustrated in FIG. 1.
The ability to provide up-scaling and conversion technology to the professional market is a substantial opportunity and industry need. A recent industry report by “The Hollywood Reporter” confirms that the greatest single catalyst that could push digital television forward in the U.S. after 2010 would be conversion technology having the capability to quickly, efficiently, and cost-effectively allow content producers to up-scale or up-scan their existing video content to higher resolution. This need is due in no small part to the extensive libraries of non-HD video content owned by all of the major studios and syndication/distribution companies of successful properties, like older television series and movies, such as “I Love Lucy,” “Seinfeld,” “Casablanca,” and the like, that would serve as excellent market attractors to higher resolution systems—if systems, methods, devices, software, or similar technologies existed and were available in the marketplace to enable such content owners or holders to deliver up-scaled content to the networks, quickly, easily, with high quality, and in a cost-effective manner.
The systems, processes, devices, and technologies disclosed and described herein provide numerous benefits and represent a substantial improvement over currently-known and available technologies that merely use, for example, pixel multiplication, line doubling, and edge interpolation—all acceptable and known up-scaling techniques, but techniques that have fallen short when it comes to delivering HD-like image quality—especially when compared with new video content that is created initially in high definition quality. Harnessing fractals' inherent resolution independence around this real world need represents an emerging opportunity in the areas of digital television, HDTV, broadband, and electronic cinema, among others.
In addition, improved processes for video up-scaling offer individual consumers, for example, the capability of taking home movies created using a video camera and up-scaling the images to the maximum resolutions available on their TV set or computer monitor, which could be up to 525 lines of resolution on a non-HD home TV or up to 1080 on a HDTV. This compares to about 240 lines of resolution provided by old VHS-quality tapes, on which many older home movie libraries were recorded.
The systems, methods, devices, and technologies described and disclosed herein represent an advance in the field of digital video up-scaling that delivers outstanding performance and image quality at the two extremes of how digital video is now produced and consumed—in the realms of the very small and the very large, up to 4 k or larger.
In the domain of the medium, the rampant growth of large-format flat panel plasma and LCD televisions has raised the bar on the quality of images that consumers now expect on movies and television shows watched in the home. People no longer record shows using videocassettes. Instead, Digital Video Recorders, Internet Video, Home Video Networks, On-Demand TV, and the next-generation of satellite and cable TV (known as IPTV-Internet Protocol Television) are the norm.
If the average consumer now expects a movie theater experience in his home, what kind of experience will they begin to demand at a real movie theater? That represents one of the biggest opportunities for digital video ever. In the domain of the very large, the systems, methods, devices, and technologies described and disclosed herein raise the bar on the level and quality of resolution that is available in movie theatres. The movie industry has been undergoing a massive shift that has been years in the making to be the next frontier in which digital changes everything. The systems, methods, devices, and technologies described and disclosed herein provide the capability to deliver pristine cinematic image quality and high performance in the growing industry of Digital Cinema. The systems, methods, devices, and technologies described and disclosed herein put large-format cinematic resolution digital video on the silver screen without any of the compromises and high costs that have slowed the digital movie theater revolution.
The following is a quick and high level (but non-exhaustive) list of features and improvements offered by the systems, methods, devices, and technologies described and disclosed herein that are believed to be novel and nonobvious, whether used alone or in combination with each other, and when compared with existing technologies known to the inventors:                1) use of a de-correlating color transform prior to computing domain range block distances;        2) use of JPEG-2000 reversible Color Transform to avoid loss of data in progressing from RGB→YUV;        3) comparison of pixels using only luminance information, although chrominance is also used in preferred embodiments;        4) artifact filtering;        5) choice of a domain block neighborhood that is off-center to improve performance on lines and certain edges including use of the distance to decide when to use it;        6) adding to the set of candidate range blocks in the case of video blocks from a previous frame;        7) adding blocks from a previous frame localized to the block's current location;        8) adding blocks from a previous frame motion corrected by means of information gained from a compressed representation of the video;        9) simultaneous calculation of the best p and q values to transform the block using the least squares approximation;        10) extended calculation of p and q subject to the further constraint that p lies between ½ and 1 to avoid certain artifacts in the resultant image;        11) reduced memory implementation, which reuses the results of previous multiply operations;        12) further refinement of the above techniques to maximize potential parallelism;        13) use of a final post·filter;        14) use of a post-filter (low·pass) filter whose strength depends on the degree of zooming;        15) use of “transform” methods to find the best matched block where this is a FFT or a number theoretic transform;        16) use of edge detection to guide and speed up the search for the best range block; and        17) use of a range screen containing more pixels than the domain screen (original frame) allowing range block offsets measured by fractional pixels (“superfine” screen).        
The above features and improvements, as well as additional features and aspects of the inventions described and disclosed herein and will become apparent from the following description of preferred embodiments of the systems, methods, apparatuses, technologies, and techniques.